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COMPACT  BASIS  TRIANGULARIZATION  FOR  THE  SIMPLEX  METHOD 

by  George  B.  Dantzig 

Alex  Orden  was  the  first  to  point  out  that  the  inverse  of  the  basis  in  the 
simplex  method  serves  no  function  except  as  a  means  for  obtaining  the  repre¬ 
sentation  of  the  vector  entering  the  basis  and  for  determining  the  new  price 
vector.  For  this  purpose  one  of  the  many  forms  of  "substitute  inverses" 
(such  as  the  well  known  product  form  of  the  inverse)  would  do  just  as  well,  in 
fact  may  have  certain  advantages  in  computation. 

Harry  Markowitz  was  interested  in  developing,  for  a  sparce  matrix,  a 
substitute  inverse  with  as  few  nonzero  entries  as  possible.  He  suggested 
several  ways  to  do  this  approximately.  For  example,  the  basis  could  be 
reduced  to  triangular  form  by  successively  selecting  for  pivot  position  that 
row  and  that  column  whose  product  of  nonzero  entries  (excluding  the  pivot) 
is  minimum.  He  also  pointed  out  that,  for  bases  whose  nonzeros  appear  in 
a  band  on  staircase  about  the  diagonal,  proper  selection  of  pivots  could 
result  in  a  compact  substitute  inverse  with  no  more  nonzeros  than  the  original 
basis. 

We  shall  adopt  Markowitz's  suggestion.  However,  instead  of  recording 
the  successive  transformations  of  one  basis  to  the  next  in  product  form,  we 
shall  show  that  it  is  efficient  to  generate  each  substitute  inverse  in  turn  from 
its  predecessor.  The  substitute  inverse  remains  compact,  of  fixed  size. 

Thus  "reinversions"  are  unnecessary  (e>:-ept  in  so  far  as  they  are  needed  to 
restore  less  of  accuracy  due  to  cumulative  round  off  error). 


The  procedure  which  we  shall  give  can  be  applied  to  a  general  mxm 
basis  without  special  structure.  As  such,  it  is  probably  competitive  with  the 


standard  product  form,  for  it  may  have  all  of  its  advantages  and  none  of  its 
disadvantages.  With  certain  matrix  structures,  moreover,  it  appears  to  be 
particularly  attractive. 

We  shall  focus  our  remarks  on  staircase  structures.  The  reader  will 
find  no  difficulty  in  finding  an  equally  efficient  way  to  compact  block-angular 
structures.  Letting  be  a  submatrix  of  the  basis,  a  basis  B  with  stair¬ 

case  structure  has,  for  example,  the  form: 

(1)  B  = 


e 

In  (2),  the  marks  x,  *  ,  indicate  the  staircase  pattern  of  nonzero 

entries  in  the  basis-matrix  B.  P  is  some  column  of  coefficients  not  in 

s 

the  basis.  The  asterisks  along  the  diagonal  mark  the  successive  pivot 
positions.  It  is  assumed  (and  this  need  not  be  true)  that  the  basis  can  be 
reduced  to  triangular  form  by  pivoting  successively  on  the  lower  right-hand 
element  of  each  submatrix  formed  by  deleting  the  preceding  pivot  row  and 
column.  Each  pivot  operation  consists  in  using  the  assumed  nonzero 
diagonal  term  to  eliminate  the  column  variable  from  all  nonzero  terms 
above  the  diagonal  only.  The  symbol  indicates  the  resulting  position 

of  zero  coefficients  above  the  diagonal. 


(^^op)  (enter) 


Let  T  be  the  resulting  triangular ized  matrix;  it  has  the  form  (3). 
Note  particularly  that  the  pattern  of  nonzeros  in  T  is  precisely  the  same 
as  the  pattern  of  nonzeros  on  and  below  the  main  diagonal  of  the  original 
basis  B  and  that  ,  the  transform  of  under  the  same  row  operations, 
may  have  nonzeros  in  its  leading  components. 

♦ 

X  * 

X  X  * 

X  X  X  * 

X  * 

X  X  * 

(drop) 

Since  T  is  obtained  from  B  by  a  sequence  of  operations  on  rows, 
this  is  the  same  as  multiplying  B  on  the  left  by  a  succession  of  elementary 
matrices  so  that 

. 


(enter) 
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Here  represents  an  elementary  matrix  corresponding  to  a  pivot  in 

row  6.  Thus  the  first  pivot  operation  is  the  same  as  multiplying  B  on  the 
left  by 


(5) 


^6  = 


56 


where  p^^  is  selected  so  that  row  6,  when  multiplied  by  p^^^  and  added  to 
row  5,  will  cause  the  element  (5,6)  of  the  matrix  to  vanish.  Since  no 
eliminations  are  required  in  column  5,  E^  is  an  identity  matrix.  Next,  E^ 
will  be  similar  to  E^  except  with  one  nonzero  entry  p^^  for  element  (3,4), 
E^  will  have  at  most  two  nonzero  entries  above  the  diagonal  Pj3« 
corresponding  to  the  factors  required  to  eliminate  elements  (1,  3)  and  (2,  3) 
from  the  matrix  using  row  3.  Similarly  E^  will  have  an  entry  p^^* 
will  be  an  identity  matrix.  Since  each  elementary  matrix  E.  is  an  identity 
matrix  except  for  nonzero  entries  above  the  diagonal  of  column  i,  we  may, 
for  purposes  of  compact  recording,  simply  list  side  by  side  the  entries  in 
column  1  of  E^,  in  column  2  of  E^,  etc.  We  shall  refer  to  this  typical 
product  form  record  of  the  transformations  as  the  E-structure.  For  our 
example 


(6) 


E  (Structure) = 


1 

Pl2 

r  1 

Pl3 

1 

P23 

1 

P34 

1 

1 

P56 

. 

-  J 

.  J 

.  J 

1 
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Note  again  that  the  pattern  of  nonzeros  in  the  E- structure  (excluding  the  units 
on  the  diagonal)  is  precisely  the  same  as  the  pattern  of  nonzeros  above  the 
main  diagonal  of  the  original  basis  B.  Thus  the  statement  in  product  form 
of  the  nonzero  coefficients  in  the  transformations  necessary  to  reduce  a 
basis  to  triangular  form  T  and  the  record  of  nonzeros  in  T  have  as  compact 
a  representation  as  the  original  basis. 

We  give  the  formulae  for  the  determination  of  the  set  of  simplex 
multipliers  (or  pricing  vector)  it  and  the  representation  of  the  vector 
Pg  entering  the  basis.when  E^  and  T  are  known.  Let  y  be  the  vector  of 
coefficients  of  the  cost  form  for  the  basic  variables,  then  by  definition 

(7)  IT  B  =  y  . 

If  now  we  define  it*  by  the  relation 

(8)  TT*T  =  y 

then,  it  is  easy  to  see,  by  (4),  that 

(9)  IT  =  tt*E,E,,  .  .  .  ,  E  . 

I  c  m 

Because  T  is  triangular,  tt*  can  be  directly  computed  from  (8)  and  tt 
from  (9)  by  applying  to  it*  the  transformations  E^  E^,  ...  in  turn  on 
the  right. 

Having  obtained  tt,  we  can  by  the  usual  "pricing  out"  procedure 
determine  the  vector  to  enter  the  basis  by 

(10)  ttP  =  Min  ttP.  <  0  . 

s  •  J 

J 
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By  definition,  the  representation  P  of  in  terms  of  the  basis  satisfies 

a  S 

(11)  BP  =  P  . 

s  s 

If  now  we  define  P*  by  the  relation 
s 

=  . 

then,  it  is  again  easy  to  see, by  (4)  and  (11),  that 

(13)  TP  =  P*  . 

s  s 

Pg  is  easily  computed  by  (12)  and,  because  T  is  triangular,  P^  is 
computed  by  direct  solution  of  (13). 

Given  a.nd  the  b3.sic  fe3.sible  solution^  the  usu3.1  rules  are  next 
applied  to  determine  the  vector  P^  to  drop  from  the  basis  and  to  determine 
the  basic  feasible  solution  for  the  next  iteration.  We  shall  omit  these  steps 
assuming  they  are  known  to  the  reader. 

Our  problem  now  becomes  one  of  "up-dating"  our  substitute  inverse. 
This  of  course  could  be  done  by  succession  of  pivot  operations  above  the 
diagonal  such  as  we  described  earlier.  But  this  is  not  very  efficient 
computationally.  We  shall  show  instead  an  efficient  procedure  for  easily 
modifying  the  E- structure  and  T  matrix  of  one  iteration  to  obtain  those 
of  the  next. 

Let  us  assume  in  our  example,  see  (2),  that  the  vector  P^  entering 
the  basis,  if  entered  in  its  proper  position  in  the  staircase  array,  would  be 
located,  say,  between  either  vectors  P^  and  P^  of  the  basis  (or  vectors 
P^  and  P^),  and  let  us  suppose  that  vector  P^  is  to  be  dropped.  Starting 
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with  the  columns  of  B  and  after  they  have  been  transformed  by  the  row 
operations  E^E^,  namely  with  T  and  P*  as  shown  in  (3),  our 

objective  is  to  triangularize  the  matrix  formed  by  deleting  the  first  column 
and  introducing  P^  ,  say,  between  columns  4  and  5  (actually  between  columns 
3  and  4  would  be  less  work).  The  row  operations  that  accomplish  this  are  to 
create  zeros  in  the  first  three  rows  of  P^  column  by  successively  adding 
first  a  multiple  of  row  2  to  row  1,  next  a  multiple  of  row  3  to  row  2,  and 
then  a  multiple  of  row  4  to  row  3.  We  shall  denote  these  single  row  trans- 
formations  by  E^,  E^,  E^.  For  the  present  we  have  assumed  above  that  the 
second,  third  and  fourth  components  of  P^  are  nonvanishing  (this  need  not 
be  the  case).  The  results  of  these  operations  are  shown  in  (14)  where  * 
indicate  the  elements  of  the  previous  diagonal  and  a  those  of  the  new  diagonal. 


The  relationship  between  the  new  T  and  the  new  B  may  be  written 
(15)  (New  T)  =  e|e3E2EjE2E3E4E5E^  (New  B)  . 

K,  however,  the  column  to  be  dropped  were  j  =  4  (instead  of  j  =  1),  it  would 
be  necessary  to  eliminate  the  D  element  in  column  3  and  then  the  ones  in 


column  2  by  additional  transformations  of  the  type  ,  say  E^,  E^  ,  in 

this  case 

(New  T)  =  e]e^E^E^E2E^E2E3E4E5E^  (New  B)  . 

We  have  shown  that  the  new  T  can  be  obtained  by  applying  to  the 
previous  product  of  the  E^  a  succession  of  row  operations  of  the  form 
e!  where,  in  general,  we  have  denoted  by  E^^  an  elementary  matrix 
corresponding  to  adding  a  multiple  of  row  i  to  row  k.  Our  objective, 
however,  has  not  been  accomplished  until  we  have  shown  how  to  obtain 
easily  the  new  T  directly  from  the  new  B  by  a  succession  of  new  pivot 
operations  .  This  is  easy  to  accomplish  if  we  observe  the  following 
rules: 

I.  If  and  E^  are  two  elementary  matrices  representing  adding 
a  multiple  of  row  i  to  other  rows,  then  their  product  E.E.  can  be  replaced 
by  an  elementary  matrix  of  the  same  type,  say.  E*  .  For  example 


• 

•  . 

1  Pl3 

1  Pl3 

^  Pl3+Pl3 

^  P23 

^  P23 

= 

^  P23, +  P23 

1 

1 

1 

II.  "Near  commutativity"  of  adjacent-indexed  matrices  E^‘^  and  E 

1  i- 

holds;  thus  the  product  e!  ^  can  be  replaced  by  E.  j^E^.  For  example 


1 

1  qi3 

1  qi3 

^  -‘113P34 

1 

^  ^123 

= 

^  ^123 

^  -‘123P34 

^  P34 

1 

1 

^  P34 

1 

L 

1_ 

1 
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III.  Nonadjacent- indexed  matrices  can  be  commuted;  thus 


Note  particularly  that  the  formation  of  each  E.  ,  from  a  computational  point 
of  view,  consists  essentially  of  rr.ultiplying  most  of  the  elements  of  column 
i  -  1  of  ^  by  a  constant  and  adding  it  to  the  corresponding  elements  of 
column  i  of  E.. 


The  process  described  above  of  reducing  to  triangular  form  the  matrix 
formed  by  dropping  a  column  of  T  and  inserting  was  based  on  the 

assumption  that  certain  coefficients  of  were  nonzero.  If,  for  example, 

the  second  component  is  zero  but  the  first  component  is  not,  it  would  not  be 
possible  to  use  a  row  operation  E^  to  cause  the  first  component  of  P^  to 
vani  sh. 


Let  us  suppose  that  the  position  of  column  P^  in  the  new  basis  is 
k  columns  from  the  left  (we  assume  that  pivoting  is  done  by  starting  always 
with  the  lower  right-hand  element  of  each  submatrix). 


Let  the  column  being  dropped  from  the  basis  be  r  <  k.  In  the  process 
of  computation  of  P*  by  (12),  we  obtain  the  vector 


=  %^k+l . ^m^s  • 

Now  P'  must  have  its  first  nonzero  component  for  some  index  h  <  k  since 

s  th  * 

the  new  B  is  nonsingular.  We  assume  ior  the  moment  that  the  k  componen 

is  not  aero.  Accordingly,  starting  with  ,  the  elimination  o£  its  first  non- 
aero  component  with  index  s ,  can  be  efiected  by  using  its  second  nonaero 

component  with  index  s^,  etc.  until  its  nonaero  component  with  index  s,  =  k 

is  used.  This  corresponds  to  row  operations  o£  the  £orm  tollowed  by 

£^2  ^  remaining  components  with  indices  k,  k+1 . m  are  un  ^ 

affected  by  the  above  operations  and  hence  remain  the  same  as  those  of  P^  • 

Thus  the  result  is  the  same,  as  far  as  columns  Pi^.I\+l . ^m  concerne 

as  if  triangularization  had  been  effected  directly  using  these  columns.  If 

+  1  .  =  A>  0,  it  will  be  necessary  to  cyclically  permute  certain  of  the  rows 

by  relabeling  rows  Sj^.s^+l . ‘  ^  as  rows  s^  1.  Sj,s,+  ,  i  £ 

In  a  similar  manner,  rows  . ®3  ‘  ^  permuted  i 

S2  +  1  -  83  >  0,  etc.  Allowing  such  permutations,  it  is  no  longer  necessary  to 

assume  above  that  the  k'^  component  of  P*  (or  P'^  )  was  not  zero. 

It  is  important  to  note  that  such  permutations  would  have  been  required 
if  direct  triangularization  of  all  columns  had  been  effected  initially.  More¬ 
over,  as  far  as  staircase-structured  systems  are  concerned,  these  permuta¬ 
tions  would  not  have  affected  the  below-diagonal-staircase-form  of  T  or  the 
above-diagonal-staircase-form  of  the  E-structure  because,  if  direct 

eliminations  were  used,  the  eliminations  and  row  interchanges  would  have 
been  confined  only  to  rows  where  the  components  of  P^  are  nonzero. 
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Let  us  now  turn  our  attention  to  the  column  to  be  dropped  from  the 

basis.  Suppose  first  r  <  k.  Deletion  of  the  corresponding  column  of  T 

followed  by  the  necessary  eliminations  to  restore  triangularity  discussed 

earlier  will  also  require  permutations  if  the  indicated  pivot  position  along 

the  diagonal  has  a  zero  coefficient.  For  example,  a  two-cycle  permutation 

V  !1  be  required  in  order  to  lower  to  the  diagonal  the  nonzero  coefficient  just 

above  the  diagonal.  If  r  >  k,  it  appears  to  be  necessary  first  to  drop  the 

column  corresponding  to  from  T  and  to  retriangularize  columns 

k,  k+ 1,  .  .  .  ,  r  -  1  (omitting  r),  and  next  to  insert  the  column  corresponding  to 

P  by  performing  the  eliminations  described  above  to  P 
s  s 

Since,  in  general,  row  permutations  are  required  to  obtain  the 

triangular  arrangement  in  standard  form,  it  is  necessary  to  replace  (4)  by 

(4-)  T  =  E^E^,. 

where  J  represents  a  permutation  matrix.  Each  new  cyclic  permutation 

C  introduced  in  the  process  of  elimination  to  a  new  triangular  form  can  be 

accounted  for  by  appropriately  relabeling  the  row  designations  of  coefficients 

in  E.  and  J. 

1 

Finally,  it  is  necessary  to  restate  the  rules  given  earlier  for  up-dating 
the  substitute  inverse  when  elementary  matrices  of  the  type  E^ ,  where 
i  <  i,  appear  on  the  left  instead  of  Ej  ^  discussed  earlier.  In  this  case 
the  rules  are: 

II':  E^E^  =  where  i  <  i  and  where,  letting  p^  be  the 

i,  i  component  of  E^ ,  the  i^^  column  of  is  formed  by  multiplying  by  -p^. 
the  corresponding  coefficients  in  column  i  of  E^  with  the  exception  of  rows 
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i  and  i.  For  row  i,  the  coefficient  is  ,  and  for  row  i,  the  coefficient 
is  unity. 


Ill':  E^E.  =  E.E^  if 


i  <  i  <  k 


IV :  F.E  =  E  E.  if 
1  q  q  1 


i  <  q  <  i  where  i  is  the  same  I  as 


that  which  generated  E^  in  II'  .  Note  that  the  commutativity  of  the 
matrices  holds  because  E^  has  a  zero  coefficient  in  column  i  for  row  q 
and  E  has  a  zero  coefficient  in  column  q  for  row  i. 

q 
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